Skip to content

fix(elastic+pipeline): keystore-backed cluster credentials + ProcessorsEquals self-compare - #407

Merged
medcl merged 3 commits into
mainfrom
fixes/keystore-and-reload
Aug 25, 2026
Merged

fix(elastic+pipeline): keystore-backed cluster credentials + ProcessorsEquals self-compare#407
medcl merged 3 commits into
mainfrom
fixes/keystore-and-reload

Conversation

@medcl

@medcl medcl commented Aug 25, 2026

Copy link
Copy Markdown
Member

Two framework fixes found and verified in live LogPilot/Gateway use (full multi-instance isolation testing).

1. Cluster credentials never survived the ORM (fix(elastic))

SecretString's MarshalJSON emits the shadow mask for plain-text values, so a cluster saved through the ORM (the /easysearch/ CRUD) never persisted its real basic_auth password or token — every consumer that loaded the record back (boot-time live registration, the cluster-change hook, app-side resolvers like LogPilot's stream search) authenticated with the mask and got 401s from secured clusters. The health-status persist loop made it worse: identity comparison (real in-memory vs masked ORM record) re-registered the live client with the mask.

  • StashClusterSecrets/FromDelta store plain credentials under cluster-scoped keystore keys before the ORM write; switching auth mode drops the other key so hydration cannot resurrect a stale secret
  • HydrateClusterSecrets fills masked/missing credentials wherever an ORM-loaded record is used (CRUD post hooks, cluster-change hook, LoadClustersFromORM); in-memory real values always win
  • RemoveClusterSecrets cleans up on delete; keystore.DeleteValue added
  • ORM records keep the mask, so API responses never leak the secret

2. Pipeline config reload never fired (fix(pipeline))

ProcessorsEquals built targetCfg from this instead of target — it compared the source config with itself, so Equals always saw identical processors and config-file updates never reloaded pipeline tasks. Every change (ship endpoints, harvest patterns) required a process restart. Regression test covers a nested ship_config.endpoints change.

Verification

  • Unit tests: core/elastic (cluster-secrets ORM round-trip: stash → masked marshal → hydrate → retyped password → update → delete), core/pipeline (nested-change detection), modules/elastic, modules/easysearch
  • Live end-to-end on LogPilot: secured Easysearch sink cluster delivered to gateways (credential in gateway keystore, ORM/API masked), 300k+ docs written; agent task hot-reload verified without process restart

Note: two sibling fixes that touch modules/configs/{server,reverseclient} (pending-instance sync wiping local configs; loopback rewrite breaking specific-IP bindings) are added to #405 since that package only exists there.

@medcl
medcl force-pushed the fixes/keystore-and-reload branch from dfb4a45 to 37d2be9 Compare August 25, 2026 02:29
medcl added 2 commits August 25, 2026 11:04
SecretString's MarshalJSON emits the shadow mask for plain-text values,
so a cluster saved through the ORM (the /easysearch/ CRUD) never
persisted its real basic_auth password or token — every consumer that
loaded the record back (boot-time live registration, the cluster-change
hook, app-side resolvers) authenticated with the mask and got 401s from
secured clusters.

- StashClusterSecrets/FromDelta store plain credentials under
  cluster-scoped keystore keys before the ORM write; switching auth mode
  drops the other key so hydration cannot resurrect a stale secret
- HydrateClusterSecrets fills masked/missing credentials wherever an
  ORM-loaded record is used (CRUD post hooks, cluster-change hook,
  LoadClustersFromORM); in-memory real values always win
- RemoveClusterSecrets cleans up on delete; keystore.DeleteValue added
- ORM records keep the mask, so API responses never leak the secret
targetCfg was built from this instead of target, so Equals always saw
identical processor configs and config-file updates never reloaded
pipeline tasks — every change required a process restart to take
effect (stale ship endpoints, stale harvest patterns).

Regression test covers a nested ship_config.endpoints change.
@medcl
medcl force-pushed the fixes/keystore-and-reload branch from 37d2be9 to 2b9d549 Compare August 25, 2026 03:04
@medcl
medcl merged commit 1a5ba88 into main Aug 25, 2026
5 checks passed
@medcl
medcl deleted the fixes/keystore-and-reload branch August 25, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants